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Abstract. For any finite coloured graph we define the empirical neighbourhood mea- 
sure, which counts the number of vertices of a given colour connected to a given 
number of vertices of each colour, and the empirical pair measure, which counts the 
number of edges connecting each pair of colours. For a class of models of sparse 
coloured random graphs, we prove large deviation principles for these empirical mea- 
sures in the weak topology. The rate functions governing our large deviation principles 
can be expressed explicitly in terms of relative entropies. We derive a large deviation 
principle for the degree distribution of Erdos-Renyi graphs near criticality. 



1. Introduction 

In this paper we study a random graph model where each vertex of the graph carries a random symbol, 
spin or colour. The easiest model of this kind is that of an Erdos-Renyi graph where additionally each 
vertex is equipped with an independently chosen colour. The more general models of coloured random 
graphs we consider here allow for a dependence between colour and connectivity of the vertices. 

With each coloured graph we associate its empirical neighbourhood measure, which records the number 
of vertices of a given colour with a given number of adjacent vertices of each colour. From this quantity 
one can derive a host of important characteristics of the coloured graph, like its degree distribution, 
the number of edges linking two given colours, or the number of isolated vertices of any colour. The 
aim of this paper is to derive a large deviation principle for the empirical neighbourhood measure. 

Our motivation for this project is twofold. On the one hand one may consider the coloured random 
graphs as a very simple random model of networked data. The data is described as a text of fixed length, 
consisting of words chosen from a finite dictionary, together with a random number of unoriented edges 
or links connecting the words. Large deviation results for the empirical neighbourhood measure permit 
the calculation of the asymptotic number of bits needed to transmit a large amount of such data with 
arbitrarily small error probability, see |DA06ai lDAQ6b] where this idea is followed up. 

On the other hand we are working towards understanding simple models of statistical mechanics 
defined on random graphs. Here, typically, the colours of the vertices are interpreted as spins, taken 
from a finite set of possibilities, and the Hamiltonian of the system is an integral of some function with 
respect to the empirical neighbourhood measure. As a very simple example we provide the annealed 
asymptotics of the random partition function for the Ising model on an Erdos-Renyi graph, as the 
graph size goes to infinity. 
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To be more specific about our model, we consider coloured random graphs constructed as follows: In 
the first step each of n fixed vertices independently gets a colour, chosen according to some law \jl on 
the finite set X of colours. In the second step we connect any pair of vertices independently with a 
probability p(a, b) depending on the colours a, b G X of the two vertices. This model, which comprises 
the simple Erdds-Renyi graph with independent colours as a special case, was introduced by Penman 
in his thesis |Pe98j . see |CP03| for an exposition. 

Our main concern in this paper are asymptotic results when the number n of vertices go to infinity, 
while the connection probabilities go to zero of order 1/n. This leads to an average number of edges of 
order n, the near critical or sparse case. Our methods also allow the study of the sub- and supercritical 
regimes. Some results on these cases are discussed in |DA06al lDA06b] . 

Recall that a rate function is a non-constant, lower semicontinuous function / from a polish space Ai 
into [0, oo], it is called good if the level sets {/(m) < x} are compact for every x € [0, oo). A functional 
M from the set of finite coloured graphs to M. is said to satisfy a large deviation principle with rate 
function / if, for all Borel sets B C A4, 

- inf I(m) < liminf ±logP„{M(X) € B\ 

meintB n— »oo n L ' 

< limsupilogP n {M(X) G B\ < - inf J(m) , 

mco m£cl B 

where X under P n is a coloured random graph with n vertices and int B and cl B refer to the interior, 
resp. closure, of the set B. 

Apart from the empirical neighbourhood measure defined above, we also consider the empirical pair 
measure, which counts the number of edges connecting any given pair of colours, and the empirical 
colour measure, which simply counts the number of vertices of any given colour. The main result 
of this paper is a joint large deviation principle for the empirical neighbourhood measure and the 
empirical pair measure of a coloured random graph in the weak topology, see Theorem 12.11 In the 
course of the proof of this principle, two further large deviation principles are established: A large 
deviation principle for the empirical neighbourhood measure conditioned to have a given empirical 
pair and colour measure, see Theorem 12.51 and a joint large deviation principle for the empirical 
colour measure and the empirical pair measure, see Theorem 12.31 For all these principles we obtain a 
completely explicit rate function given in terms of relative entropies. 

As an example, we consider the Erdds-Renyi graph model on n vertices, where edges are inserted with 
probability p n € [0, 1] independently for any pair of vertices. We assume that np n — > c £ (0, 1). From 
our main result we derive a large deviation principle for the degree distribution, see Corollary 12.21 
This example seems to be new in this explicit form. 

2. Statement of the results 

Let V be a fixed set of n vertices, say V = {1, . . . ,n} and denote by Q n the set of all (simple) graphs 
with vertex set V = {1, ... , n} and edge set E C £ := {(«, v ) € V x V : u < u}, where the formal 
ordering of edges is introduced as a means to simply describe unordered edges. Note that for all n, 
we have < \E\ < | n(n — 1) . Let X be a finite alphabet or colour set X and denote by Q n (X) be the 
set of all coloured graphs with colour set X and n vertices. 

Given a symmetric function p n : X x X — > [0, 1] and a probability measure [i on X we may define 
the randomly coloured random graph or simply coloured random graph X with n vertices as follows: 
Assign to each vertex v € V colour X(y) independently according to the colour law \x. Given the 
colours, we connect any two vertices u, v G V, independently of everything else, with connection 



LARGE DEVIATION PRINCIPLES FOR COLOURED RANDOM GRAPHS 



3 



probability p n (X (u) , X (v)) . We always consider X = ((X(v) : v G V),E) under the joint law of graph 
and colour and interpret X as coloured random graph. 

We are interested in the properties of the randomly coloured graphs for large n in the sparse or 
near critical case, i.e. we assume that the connection probabilities satisfy np n (a,b) — > C(a,b) for all 
a, b G X, where C : X x X — > [0, oo) is a symmetric function, which is not identically equal to zero. 

To fix some notation, for any finite or countable set y we denote by M(y) the space of probability 
measures, and by M(y) the space of finite measures on y, both endowed with the weak topology. By 
M(y) we denote the space of counting measures on y, i.e. those measures taking values in N U {0}, 
endowed with the discrete topology. Finally, we denote by M.*(y x y) the subspace of symmetric 
measures in M(y x y). 

With any coloured graph X = ((X(v) : v £V),E) with n vertices we associate a probability measure, 
the empirical colour measure L 1 G M(X), by 

Ll ( a ) '■= ~^2 s X(v)(fl), foraeX, 
n vev 

and a symmetric finite measure, the empirical pair measure L 2 G Ai*(X x X), by 
L 2 {a,b) := - ^ [$(x(v),x(u)) + $(x(u),x(v))](a,b), foia,beX. 

(u,v)<=E 

The total mass \\L 2 \\ of L 2 , is 2\E\/n. Finally we define a further probability measure, the empirical 
neighbourhood measure M G A4(X x J\f(X)), by 

M(a,£) :=-J2 S (X(v)Mv))( a , £ )i for (M) G # xjV(#), 
n vev 

where L{v) = (l v (b), b G X) and is the number of vertices of colour b connected to vertex v. 
For every v G A4(X x N(X)) let z/i, ^2 be the Af-marginal, respectively the J\f(X)— marginal, of the 
measure v. Moreover, we define a measure {v(-,£), £(■)) G Ai(X x A?) by 

£),£(■)) (a, b) := ^ v(a,£)£(b), for a,beX. 

Define the function $: //*(* x M(X)) -► TW(A') x /St (A? x A") by $(1/) = *(•))), and 

observe that $(M) = (L 1 , L 2 ), if these quantities are defined as empirical neighbourhood, colour, and 
pair measures of a coloured graph. Note that while the first component of $ is a continuous function, 
the second component is discontinuous in the weak topology. 

To formulate the large deviations principle, we call a pair of measures (w, v) G Ai(X x X) x M(X x 
M{X)) sub- consistent if 

£),£(■)) (a, b) <zv(a,b), for all a, b G X, (2.1) 

and consistent if equality holds in (j2.1|) . Observe that, if v is the empirical neighbourhood measure 
and w the empirical pair measure of a coloured graph, (w, v) is consistent and both sides in (|2.1|) 
represent 

~ (1 + l{a=f>}) tl{edges between vertices of colours a and 6}. 
Recall the definition H{u || //) = ^ agiV cj(3;) log(a;(x)///(x)) of the relative entropy of a probability 
measure a; G .M(A') with respect to G A^(Af) and, for a measure G //f*(A? x Af) and a measure 
w G .M(Af), define 

fjc(zi7 II := H{w I (g> o>) + \\Clo ® a; || — ||ro|| , 
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where the measure Cuj ® uj G Ai(X x X) is defined by Cw (g> w(a, 6) = C(a, b)ui{a)u{b) for a, 6 G X. 
It is not hard to see that SjcfaWuj) > 0. For every (za,v) G M*{X x X) x M(X x N(X)) define a 
probability measure Q = Q\pv> v\ on. X x N(X) by 

be* ^ 

We have now set the stage to state our principal theorem, the large deviation principle for the empirical 
pair measure and the empirical neighbourhood measure. 

Theorem 2.1. Suppose that X is a coloured random graph with colour law [i and connection probabili- 
ties p n : XxX — > [0,1] satisfying np n (a,b) — > C(a,b) for some symmetric function C : XxX — > [0, oo). 
Then, as n — > oo, the pair (L 2 , M) satisfies a large deviation principle in A4*(X x X) x Ai(X x N(X )) 
with good rate function 

H(v || Q) + H(ui || //) + | i3c(ro || ^l) (w, ^) sub- consistent, 
oo otherwise. 



J(w, v) 



Remark 1 The rate function can be interpreted as follows: J{w , v) represents the cost of obtaining 
an empirical pair measure w and an empirical neighbourhood measure v. This cost is divided into 
three sub-costs: 

(i) H{y\ || \i) represents the cost of obtaining the empirical colour measure v\, this cost is non- 
negative and vanishes iff v\ = \x, 

(ii) ^f)c(ro || v\) represent the cost of obtaining an empirical pair measure w if the empirical 
colour measure is v\ , again this cost is nonnegative and vanishes iff w = C v\ (g) v\ , 

(iii) H{y \\ Q) represents the cost of obtaining an empirical neighbourhood measure v if the em- 
pirical colour measure is u\ and the empirical pair measure is w, this cost is nonnegative and 
vanishes iff v = Q. 

Consequently, J{w , u) is nonnegative and vanishes iff w = C v\ ® v\ and 

u(a,£) = M (a) TT e -(WVW) (C(a,b)/,(b)y( b ^ ^ ^ ^ ^ ^ 

This is the law of a pair (a,£) where a is distributed according to fi and, given the value of a, the 
random variables £(b) are independently Poisson distributed with parameter C(a,b)f/,(b). 

Remark 2 Our large deviation principle implies individual large deviation principles for the measures 
L 2 and M by contraction, see |DZ981 Theorem 4.2.1]. Note that, by the discontinuity of the 
functional relationship L 2 = $%(M) may break down in the limit, and hence the rate function may be 
finite on pairs which are not consistent. 

As usual, the degree distribution D € A4(N U {0}) of a graph with empirical neighbourhood measure 
M is defined by 

D ( k ) = J2 E hfcbexW)) M M> for A: GNU {0}. 

As the degree distribution D is a continuous function of M, Theorem 12 . 1 1 and the contraction principle 
imply a large deviation principle for D. For a classical Erdds-Renyi graph the rate function takes on 
a particularly simple form. 
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Corollary 2.2. Suppose D is the degree sequence of an Erdos-Renyi graph with connection probability 
p n G [0,1] satisfying np n — > c G (0, oo). Then D satisfies a large deviation principle, as n — > oo, in 
the space M(NU {0}) wrei/i good rate function 

\ ii log (|) -±x + f + tf(d || fe ), if(d)<c, 
where x = x(d) is the unique solution of 

-2(1- &) 

x = ce V 31 / , 

where q\ is a Poisson distribution with parameter A and (d) := X^m=o m ^( m )' 

Remark 3 On probability measures d with mean c the rate simplifies to the relative entropy of d 
with respect to the Poisson distribution of the same mean. In BGL02, Theorem 7.1] a large deviation 
principle for the degree sequence is formulated for this situation, albeit with a rather implicitly defined 
rate function. Moreover, the proof given there contains a serious gap: The exponential equivalence 
stated in BGL02, Lemma 7.2] is not proved there and we conjecture that it does not hold. 

We now state the two large deviation results, Theorems 12.31 and 12.51 which are the main ingredients 
for our proof of Theorem I2.1[ but are also of independent interest. The first of these is a joint large 
deviation principle for the empirical colour measure L 1 and the empirical pair measure L 2 , the second 
a large deviation principle for the empirical neighbourhood measure M given L 1 and L? . 

Theorem 2.3. Suppose that X is a coloured random graph with colour law \x and connection prob- 
abilities satisfying np n (a,b) — > C(a,b) for some symmetric function C : X x X — > [0, 00). Then, as 
n — > 00, the pair {L\L 2 ) satisfies a large deviation principle in A4(X) x Ai*(X x X) with good rate 
function 

I(uj,w) = H(uj\\fi) + ±f)c{™\\u). (2.3) 

Example 1 We look at the Erdos-Renyi graph with connection probabilities p n satisfying np n — > c € 
(0, 00) and study the random partition function for the Ising model on the graph, which is defined as 



:= ^2 ex P \P ^2 r l( u ) r l( v )j f° r t ne inverse temperature (3 > 0. 
v e{-i,+i} v (u,v)eE 
Denoting by E expectation with respect to the graph, we note that 



where E is expectation with respect to the graph randomly coloured using independent colours chosen 
uniformly from {—1, 1}. Then Varadhan's lemma, see e.g. 0HOO ( III. 3], Theorem 12.31 gives 

lim ilogEZ(/3) 

n—>oo " 

= log2 + sup|f J xyvu(dxdy) - I(uj,w) : uj G M(X), w G M*({-1, 1} x {-1, l})j (2.4) 

= sup 1 1 (ro(A) - w(A c )) - xlog(x) - (1 - x)log(l - x) - \ {H{w \\uo x ) + c - , 

where A is the diagonal in { — 1,1} x { — 1,1}, and the supremum is over all x G [0,1] and w G 
M*({— 1, 1} x {—1, 1}), and the measure uo x G M*({— 1, 1} x {—1, 1}) is defined by 

u x {i,j) = cx( 2+i+j V 2 (l - xf 2 '^! 2 for i,j G {-1, 1} . 
Note that the last expression in Q2.4JI is an optimisation problem in only four real variables. 
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Corollary 2.4. Suppose that X is a coloured random graph with colour law \x and connection prob- 
abilities satisfying np n (a,b) — > C(a,b) for some symmetric function C : X x X — > [0, oo). Then, as 
n — ► oo, the number of edges per vertex \E\Jn satisfies a large deviation principle in [0, oo) with good 
rate function 

C(x) = xlogx — x + inf {tp(y) — xlog(iy) + iy}, 

y>0 

where ip(y) = inf H(uj \\ fi) over all probability vectors uj with lu t Cu> = y. 

Remark 4 In the Erdos-Renyi case C(a,b) = c one obtains ip{y) = for y = c, and ip(y) = oo 
otherwise. Hence C( x ) = xlogx — x — xlog(|) + §, which is the Cramer rate function for the Poisson 
distribution with parameter |. In |BP03| a large deviation principle for \E\/n 2 is proved for coloured 
random graphs with fixed connection probabilities. 

For any n G N we define 

M n {X) := {uj G M{X) : nu(a) G N for all a£X}, 
M n {X x X) := [w G M*(X x A') : 1+1 ™ a=b y ro(o, b) G N for all o,k^}, 

We write 

SPt[cj,€t7] = {z/: (ro,z/) sub-consistent and z^i = C Ai(X x TV^)). 

Theorem 2.5. Suppose the sequence (oj n ,w n ) G A4 n (<Y) x M. n (X x ^f) converges to a limit (u,zu) G 
.M(<-f) x A4*(X x A'). Zei X be a coloured random graph with n vertices conditioned on the event 
{^(M) = (uj n ,'aj n )}. Then, as n — > oo, the empirical neighbourhood measure M of X satisfies a large 
deviation principle in the space M.(X x Af(X)) with good rate function 

W-) = P H9) ^ e ^'H (2.5) 
1 ' jV y [ oo otherwise. v ; 

In the remainder of the paper we give the proofs of the results set out so far. Section |H1 is devoted 
to the proof of Theorem \2.'A\ which uses a change-of-measure technique. By contrast, the proof of 
Theorem 12.51 carried out in Section 0J is based on combinatorial arguments combined with a fairly 
sophisticated probabilistic approximation technique. In Section E] we combine these results to obtain 
our main result, Theorem l2.1[ using the setup and result of Biggins _Bi04 to 'mix' the large deviation 
principles. The paper concludes with the proofs of Corollaries 12 . 21 and 12 A\ which are given in Section H3 

3. Proof of Theorem 12.31 by change-of-measure 
Given a function / : X — > R and a symmetric function g : X x X — > R we define the constant Uj by 

^/ = log^e /(a V(a), 

and the function h n : X x X — > R by 

h n (a, b) = log [(1 - Pn (a, b) + Pn (a, b)e^) ~ n ] , (3.1) 

for a,b G X. We use / and g to define (for sufficiently large n) a new coloured random graph as follows: 

• To the n labelled vertices in V we assign colours from X independently and identically ac- 
cording to the colour law fx defined by 

£( a ) = e /(a) ~%(a). 



LARGE DEVIATION PRINCIPLES FOR COLOURED RANDOM GRAPHS 



7 



Given any two vertices u, v € V, with u carrying colour a and v carrying colour b, we connect 
vertex u to vertex v with probability 

p n {a,b)e^ 



Pn(a,b) 



l-p n (a,b) + Pn (a,b)e9( a > b y 



We denote the transformed law by P. We observe that jl is a probability measure and that 
absolutely continuous with respect to P as, for any coloured graph X = ((X(v): v € V),E), 



is 



^ m _ TT PW 
_ 11 M*W) 



n 



p n (A»,A») TT l-p„(X(n),X(^)) 
p„(X(u),X(v)) 11 l-p n (X(«),A-(t;)) 
(u,v)&E (u,v)&E 



n n(x(u)) 



Pn(X(u),X(v)) v n-np n (A»,A») 
X 



n n-np n (X(u),X(v)) 
n-np n (X(u),X(v)) 



II p„(X(u),A») A n-np n (A»,A») 

= Y[e f ~ {x{u)) ~ u f Jl e 9inu)Mv)) e ^hn(x(u),x(v)) 

u&v (u,v)eE (u,v)e£ 

= exp (n(L\ / - Uf) + n(\L 2 ,g) + n(\l} ® L\h n ) - (\L\,h n )) , (3.2) 

I E ue y 5 (X(«),A( U )) and we denote (5,07) := Ea,te* ff( a > fo ) ro O> h ) for ro G x x ), 

^2a&x f( a ) u ( a ) f° r w € AI(A'). The following lemmas will be useful later. 

Lemma 3.1 (Euler's lemma). If np n (a,b) — > C(a,b) for every a,b € Af, i/ien 

lim [1 + op n (a,6)] n = e aC ( a ' fe ), for all a,b£X andaeR. (3.3) 



where L 1 ^ 
and (/, w) 



Proof. Observe that, for any e > and for large n we have 



1 + 



»C(a,6)— e 



< 



1 + ap n (a,b) 



< 



1 + 



aC(a,fr)+£ 



by the pointwise convergence. Hence by the sandwich theorem and Euler's formula we get (|3.3I) . 



Lemma 3.2 (Exponential tightness). For every 9 > f/iere exists JV£N suc/i i/iai 

limsup±logP{|£| > raivj < -9. 

Proof. Let c > max,, ^^ C(a, 6) > 0. By a simple coupling argument we can define, for all 
sufficiently large n, a new coloured random graph X with colour law fi and connection probability — , 
such that any edge present in X is also present in X. Let be the number of edges of X. Using 
Chebyshev's inequality, the binomial formula, and Lemma 13. 11 we have that 

n(n — 1) 

k (n(n-l)/2\ (c\k( c \n(n-l)/2-k 



»{\E\>nl}<e~ nl E{eW} = e~ nl £ e k ( n ( n ~ l ) l 2 \ 

k=0 ^ ' 



- 1-- 

nJ \ n 



n nJ 

Now given 9 > choose JV£N such that N > 9 + c(e — 1) and observe that, for sufficiently large n, 

F{\E\ > nN} < F{\E\ > nN} < e~ nd , 
which implies the statement. ■ 
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3.1 Proof of the upper bound in Theorem 12.31 

We denote by C\ the space of functions on X and by Ci the space of symmetric functions on X x X, 



for (u,w) G M{X) x M{X x X). 

Lemma 3.3. For each closed set F C M(X) x M(X x X), 

limsupilogPKL 1 ,^ 2 ) € F] < - inf 

Proof. Fix / G d and g € C 2 . Define /3: * x X -> K by /3(a, 6) = (1 - e^ a < b ^)C{a, b). Observe that, 
by Lemma 13.11 (3(a,b) = lim,,^^ h n (a, b) for all o, b £ X, recalling the definition of h n from (|3.1|) . 
Hence, by (|3.2j) . for sufficiently large n, 

e ma XaeA .|/3(a,a)| > J e (\L\,h n ) ^ _ ^^(L 1 J-U f )+n(\L 2 ,g)+n(\L 1 ®L 1 ,h n ) j ^ 

where = £ E u6 y § (X(u),X(u)) and therefore, 



and define 



I(uj, zu) = sup 

9SC 2 



{ £ (/(a) - U f )u(a) + \ £ 6) w (a, 6) + ± (1 - e^)C(a, b)u{a)u;(b)} 





(3.4) 



We now fix e > 0, and let l e (u;,ro) = min{i(cj, w), e 1 } — e. Suppose that (lu,vu) £ F and observe 
that I(uj,w) > I £ (lu,tu). Choose / € C\ and g € C2 such that 



(/- Uf,u) + \{g,tu) + \ (j3,u®u) > I £ (uj,w). 
As X is finite, there exist open neighbourhoods B^ and of w, u such that 

. in f {(/- Uf,u>) + \ (g,w) + \ 0,cj® Co)} > I e {u,m) - e. 





<nm S upilogE{ e ^ 1 ^- a /) +ri ^ 2 ^ +n ^ I ' 1 ® il '^)-J £ (a; )ro )+ e (3.5) 

77, — >00 ^ ' 



n—>oo 



< —I E (u, zu) + e. 



Now we use Lemma 13.21 with M = e 1 , to choose iV(e) € N such that 




00 



(3.6) 



For this N(e), define the set K N r £ \ by 




(lo,w) G x _M(,Y x Af) : [|tu|| < 2N(e) 
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and recall that \\L 2 \\ = 2^. The set i£jv( e ) H F is compact and therefore may be covered by finitely 
many sets B^ r x B^ r ,r = 1, . . . , m with (u r ,vj r ) G F for r = 1, . . . , m. Consequently, 

m 

P{(L\L 2 ) G F) < ^P{(L\L 2 ) G < x Bl r ) + P{(L\L 2 ) Jfy e) }. 

r=l 

We may now use (|3.5j) and ()3.6j) to obtain, for all sufficiently small e > 0, 

limsupilogP^L 1 ,! 2 ) £?}< maxlimsupilogPj^L 2 ) G < x 5^} 

n^oo r—1 n^oo 

<— inf ie(a;, vj) + e. 

(aj,ro)GF 

Taking e { we have the desired statement. ■ 

Next, we express the rate function in term of relative entropies, see for example [15298., (2.15)], 
and consequently show that it is a good rate function. Recall the definition of the function / from 
Theorem IO 

Lemma 3.4. 

(i) I(uj,w) = I(oj,vj), for any (u,vj) G M.(X) x M(X x X), 

(ii) I is a good rate function and 

(iii) $)c{ w || > with equality if and only if vj = Cuj % u. 

Proof. (i) Suppose that vj Cuj ® uj. Then, there exists do, bo G X with Cuj (g) u>(ao, bo) = and 
vj(a , b ) > 0. Define g : X x X — > E by 

£(a, 6) = log [iT(ll( a0i6o )(a, 6) + l (i)0iao )(a, 6)) + l] , for a, 6 G X and K > 0. 

For this choice of g and / = we have 

£ (/(a) - L/»w(a) + £ ^(a,6Mo,6)+ £ \{l - e^)C{a,b)u(a)uj(b) 

a&X a,beX a,beX 

= 2 log(-fC + 1)-O7(ao, 6o) OO) f° r -K" T oo. 

Now suppose that zu <C Cu (g> w. We have 

I>, tu) = sup { £ (/(a) - log £ e'< a V(a)) w(a)} 



+ 2 



§ C(a,b)u(a)u(b) + \ sup { ^ g(a,b)m(a,b) - e 9(a>6) C7(a, 6)w(o)w(6)}. 



a,beA" 9£C2 a.fcGA' a,beX 

By the variational characterization of entropy, the first term equals H(u\\fj,). By the substitution 
h = e 9 the last term equals 



sup V [log (h(a, b) m a \ b ) - h(a, 6)1 w(a, b) 
h ^2 „t^v L V C(a,b)u(a)u(b)J J 



a,b&X 



= sup V ( log h(a, b) - h(a, b)) vj(a,b) + V log (— — — ) tu(a, 6) 
h ^aTtx a^x ^C(a,bMaMb)J 

= — \\vj\\ + H {vj II Cuj <g) w), 
where we have used sup a , >0 logx — x = — 1 in the last step. This yields that I(uj, vj) = vj). 
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(ii) Recall from (|2.3j) and the definition of Sjc that w ) = H(uj \\ H(vj \\ Cuo®uo^+^ ||Ca;<8>u;|| — 
5 ||ro||. All summands are continuous in oj, w and thus / is a rate function. Moreover, for all a < oo, 
the level sets {I < a} are contained in the bounded set {(to, w) € A4(X) x M(X x X) : ficfa \\ lo) < a} 
and are therefore compact. Consequently, / is a good rate function. 

(hi) Consider the nonnegative function £(x) = a; log a; — x + 1, for x > 0, £(0) = 1, which has its only 
root in x = 1. Note that 

? )c (u>\\u) = { It°9dCu®u if g := ^ > exists 
[ oo otherwise. 

Hence ficfa || > 0, and, if w = Cuj (g> w, then £ ( dCu®cj ) = £(1) = and so $jc(Cu; ® u \\ ui) = 0. 
Conversely, if Sjci^W^) = 0, then w(a,b) > implies ® cj(a, 6) > 0, which then implies 
£ o g(a, b) = and further g(a, 6) = 1. Hence w = Cuj u, which completes the proof of (hi). ■ 

3.2 Proof of the lower bound in Theorem 12.31 

Lemma 3.5. For every open set O C M(X) x M(X x X), 

liminf ilogP/fL 1 ,^) eo\>- inf I(o;,ro). 

n— >oo I J (w,ro)60 

Proof. Suppose (a;, ro) € O, with <C Cu <S> u>. Define f w : X — > E by 

/ u(o ) = / lo g^J' ifw(a)>0, 
10, otherwise. 



and <? ro : A? x X — > R by 



fe(a, 6) = { log c^mm ' if ro < a ' 6 ) > °' 
0, otherwise. 



In addition, we let j3 m (a, b) = C(a, b)(l — e^^ a ' b ^) and note that f3^(a, b) = limn^oo hzan( a , b), for all 
a, b € X where 

h^ n (a, b) = log [(1 - Pn (a, b) + Pn (a, b)e^^) ~" . 
Choose Bl, Bl open neighbourhoods of uj, zu, such that x B^ C O and for all (u, w) E B^ x 

We now use P, the probability measure obtained by transforming P using the functions f w , g m . Note 
that the colour law in the transformed measure is now u, and the connection probabilities p n (a,b) 
satisfy np n (a,b) — ► uj(a,b)/(u(a)u(b)) =: C(a,b), as n — > oo. Using l|3.2|) . we obtain 

p{(l\l 2 ) go} >e{§ (X)l {(il)L2)6B i xBij} } 

uev (u,v)eE (u,v)es 

>exp(-n(/ w ,cj)-ni(5 ro ,tt7)-ni(/3 ro ,a;®w) + m-ne) xP^L^L 2 ) £ B * x B 2 ^}, 
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where m := A min ae ^ f3(a, a). Therefore, by (|3.3j) . we have 
liminf ilogPlfL 1 ,^ 2 ) G o\ 

> -(f u ,u) (^^®uj)-e + \immi}\ogf{{L\L 2 ) G x T? 2 ,}. 

The result follows once we prove that 

liminfilogPj^ 1 ,^) 6^x^1=0. (3.8) 

We use the upper bound (but now with the law P replaced by P) to prove (|3.8|) . Then we obtain 
limsupilogf>{(L\L 2 ) G (fl* x 7£) c } < - inf _/(£>,*), 

n^oo (ii,tn)e-F 

where F = (B^ x B^) c and 7(d), db) := 77(d) \\u>) + hSj^{m \\ Co). It therefore suffices to show that 
the infimum is positive. Suppose for contradiction that there exists a sequence (ui n ,w n ) G F with 
I(u} n ,w n ) | 0. Then, because 7 is a good rate function and its level sets are compact, and by lower 
semicontinuity of the mapping (u,w) i— > 7(d), ro), we can construct a limit point (o),ro) G F with 
7(a), ro) = . By Lemma 13.41 this implies 77(a) || a;) = and fjc^ro || a)) = 0, hence a) = a>, and 
tz; = Cuj ®Cj = vj contradicting (a), zu) £ F. ■ 



4. Proof of Theorem 12.51 by the method of types 

Throughout the proof we may assume that a; (a) > for all a G X . It is easy to see that the law 
of the randomly coloured graph conditioned to have empirical colour measure oj n and empirical pair 
measure w n , 

P(^„)=n - i*(M) = (w n ,w n )}, 

can be described in the following manner: 

• Assign colours to the vertices by sampling without replacement from the collection of n colours, 
which contains any colour a G X exactly nu n {a) times; 

• for every unordered pair {a, b} of colours create exactly n(a, b) edges by sampling without 
replacement from the pool of possible edges connecting vertices of colour a and b, where 

nw n (a, b) if a ^ b 
S w n (a, b) if a = b . 



n(a, b) :- 



For our proof it is convenient to introduce a numbering system, which specifies, for each {a, b}, the 
order in which edges are drawn in the second step. More precisely, the edge-number k is attached to 
both vertices connecting the k th edge. Note that the total number of edge-numbers attached to every 
vertex corresponds to the degree of the vertex in the graph. All permitted numberings are equally 
probable, with the total number of possible numberings (given the coloured graph) being 

Yl n(a,b)\ . 

{a,b} 

Denote by V(a) the set of vertices with colour a, and let Y^ a,b ^ be the j th edge drawn in the process 
of connecting vertices of colours {a, b}. Let A n {<jj n ,m n ) be the set of all possible configurations 

X = ((V(a) : aeX); (Y^ a ' b} : k = 1, . . . , n(a, b); {a, b} C X) 
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and let B n (ui n ,m n ) be the set of all coloured graphs x with = u; n and L 2 (x) = zu n . Define 

A (to n ,w n ) — > B n (io n , tt n ) as the canonical mapping which associates the coloured graph to any 
configuration, i.e. 'forgets' the numbering of the edges. Finally, define 

IC^^njVJn) := |M(x) for some x G B n (uj n , vo n )\ 

to be the set of all empirical neighbourhood measures M(x) arising from coloured graphs cc with n 
vertices with $(M(x)) = (w n ,-a7 n ). 

In our proofs we use the following form of Stirling's formula, see jFe!671 page 54]: For all n G N, 

n n e~ n <n\< (27m) 3 n n e -"+V( 12 ™) _ 



4.1 A bound on the number of empirical neighbourhood measures 

In this section we provide an upper bound on the number of measures in K,^ n \uj n ,w n ). We write m 
for the number of elements in X. 

Lemma 4.1. There exists •& = #(m) > such that, if u n G A4 n (X) and w n G M n (X x X), then 

r 2m— 1 "i 

n \uj n ,zo n ) <exp [tf(logn)(n|K||)^Tj. (4.1) 

The proof is based on counting integer partitions of vectors. To fix some notation, let 3 m = (N U {0}) m 
be the collection of (nonnegative) integer vectors of length m. For any £ G 3 m we denote by its 
magnitude, i.e. the sum of its entries. 

We introduce an ordering on 3 m such that, for any vectors 

h = (£?,..., l^) £ 2 = (4 1) ,...,4 m) ). 

we write l\ £2 if either 

(i) \\l x \\ > ||4||, or 

(ii) ||4 1| = ||4 1| and there is j G {1, . . . ,m} with lf ] = £ { 2 k) , for all fc < j, and ^ > ^ or 

(iii) 4 = 4- 

A collection [£\, . . . , 4) of elements in 3 m is an integer partition of the vector £ G 3 m , if 

^ ^ . . . 4 ^ and £1 + . . . + 4 = I. 

Any integer partition of a vector £ G 3 m induces an integer partition ||-^i||, . . . , \\£k\\ of its magnitude 
\\£\\, which we call its sum-partition. We denote by V m {€) the set of integer partitions of £. 

Lemma 4.2. There exists $ = > such that, for any £ G 3 m of magnitude n, 

r 2m- 1 

V^m if) < exp $ (log n) n 2m 

Proof. Let ^ G J m be a vector of magnitude n and (£1, . . . ,4) be an integer partition of £. We 
rewrite the partition as (ltii,i, • • • , till ; tTi2,l> • • • > m 2,fc 2 ; ■ • • j m r,i, ■ ■ ■ , ^r,k T ) such that all vectors in the 
same block (indicated by the first subscript) have the same magnitude, which we denote yi,...,y r , 
and such that y\ > ■ ■ ■ > y r > 0. Note that for the block sizes we have k\ + • • • + k r = k and that 
kxyi H h k r y r = n. 

For a moment, look at a fixed block tn^i, . . . , vtij t k ■ It is easy to see that the number of integer vectors 
of length m and magnitude yj is given by 

b(y„m) := " + ™ ~ X \ < c{m) y™" 1 . 
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Writing m J; o f° r the largest and tn^.+i for the smallest of these vectors in the ordering £=, we note 
that 

p: {0, ...,kj + 1} -> {m G 3 m : ||m|| = %■}, = rtv^, 

is a non-increasing path of length kj + 2 into an ordered set of size b(yj,m), which connects the smallest 
to the largest element. The number of such paths is easily seen to be 

b(y j ,m)+k j 
kj 

Therefore, the number of integer partitions of £ with given sum-partition (yi, ■ k - 1 -,yi, . . . ,y r , - k - r -,y r ) is 

- LJ - V kj I »1 «r>0 J- J- a.fcSN V fc / 

j'=i v J 7 E« r » i=i ^ yk=«j v 7 

To maximize the binomial coefficient over the set yk = ctj, we distinguish between the cases when 
(i) a,j < c(m)y m , (ii) dj > c(m)y m and observe that 



c(m) 2 / m - 1 + ?| < 



2c(m)y" 

y 

3 



if Oj < c(m)y m , 



a 

2 



, y , if a,- > c(m)y r 



Case (i): Using the upper bound (*) < (^) r , for r, i G N with r < i and the inequality (^y) 1 ^™ < 
y < cij < n we obtain, for some constants Co = Co(m), Ci = Ci(m), 

m— 1\ /o~fm\». , m— 1 „\ a j/y m— 1 



2c(m)y m \ f 2c(m)y 



< exp (C (aj/y) log n) < exp ((log n) Ci a • m ) . 



Case (ii): The same upper bound for binomial coefficients and 1 < y < (^)V" 1 < aj < n yield for 
some constant C2 = C2(m), 



2of/« \ / 2(o,-/!/)e V |m)!/ ™" /„ — 



m 



From this, we have for some C = C(m) > 0, the upper bound 

II ( biVj, f. +hj )^ oi - o nexp((logn)Ca^) , 
which is estimated further (using Holder's inequality) by 

exp (Jlog n) C 2J m j < exp (jlog n) C r m ^ OjJ J • 
j'=i ' j'=i 
We observe that all yj are different, positive and that their sum is not greater than n, so we have that 

r 2 /2 < 1 + . . . + r < yi + . . . + y r < n. 

Recalling that ai + ■ • • + a r = n, our upper bound becomes 

( 1 m— 1 \ /a , 2m— 1 \ 

(log n) C (2n) •» J = exp I § (log n)n z m , 



14 



KWABENA DOKU-AMPONSAH AND PETER MORTERS 



for some # = $(m) > 0. Note that from our argument so far one can easily recover the (well- 
known) fact that the number of integer partitions of n is bounded by e^/ 2 -* Combining this with 
the upper bound for the number of integer partitions with a given sum-partition, we obtain the claim. ■ 

Proof of Lemma 14.11 Suppose ui n G M n (X) and w n G M n (X x X). For a G X, we look at the 
mappings 

where (L", . . . , L^,-,) is the ordering of the vectors L(v), for all v G V with X(v) = a, and thus 
constitutes and integer partition of the vector 

(nm n {a, b) \ b G X) , 

which has magnitude n^2 b w n (a,b). The combined mapping $ = (<£ a : a G X) is injective, and 
therefore, by Lemma l4,2| 



tt/C (n) (w n ,ro n ) < exp [# ^ log (n^ro n (a, b)) (w^ro n (o,5)) 

aeA" 6eA' fee* 

r i 2m— 1 -i 

< exp i? log (n||ro||) (n||w n ||) 2m , 



2m- 1 
2m 



where we have used the fact that w n (a, b) < \\w\\ and Holder's inequality in the last step. ■ 
4.2 Proof of the upper bound in Theorem 12.51 

We are now ready to prove an upper bound for the large deviation probability in Theorem 12. 51 starting 
with a lemma based on the method of types. 

Lemma 4.3. For any sequence (u n ) with v n G IC^ n \uj n , w n ) we have 

P{M = u n | *(M) = {u n , w n )} < exp ( - nH{u n \\ Q n ) + ef\u n )) , 

where 

Q n {a, £) = uj n (a) [[ — , forleN{X), 

hex ^ >' 

and 

lim \ sup £ { i\v n ) = 0. 

Proof. The proof of this lemma is based on the method of types, see |DZ 98, Chapter 2]. For any 
v n G 

£(«) 

(uj n , w n ) we have 

F{M = u n *(Af) = (w n ,o7 n )} = -!= ^. (4.2) 

Now, by elementary counting, the denominator on the right side of (|4.2|) is 

n \ n "ff /n^W^l^) -nw„(a)l {fl=6} \ 
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For a given empirical neighbourhood measure u n G IC^ n \uJ n , w n ) the numerator is probably too tricky 
to find explicitly. However, an easy upper bound is 

)T]( n UWn{aM Wf-(A), (4 .4) 

nv n (a,l),aeX,£eAf(X)J 11 W(b), j = 1, ... ,nu n (a)J 1 ; 

where £a\b), j = 1, • • • , nu> n (a) are any enumeration of the family containing each £(b) with multiplic- 
ity nu n {a,£). This upper bound is obtained by attaching edge-numbers without discounting for the 
possibility of multiple edges or loops. In the case a = b initially edges are considered to be oriented 
and then the orientation is forgotten, leading to the extra term 2~ 2" ro ™( A ). Combining (|4.2jl . (|4.3|) . 
and (|4.4|) we get 

P{M = i/ n |*(M) = K,ro n )} 

< tt / nw«(a) \ -r-r / nro„(a,6) 

" 11 U^(a^), * € N{X)) 11 V ^(6), j = 1, . . . ,fl^(a), (45) 

n(a,ft) 

x 2 -fro„(A) J-j- ]J p 2 ^n(aVnW-n^ n (a)l {a ^ } ^ ^-1 
{a,fe} fe=l 

It remains to analyse the asymptotics of the upper bound. Using Stirling's formula, we obtain 

I! ( ( c A/7 v<t ) - exp ( n Wn (°) log Wn ( a ) ~ n Yl ^( a > ^ log ^( a ' ^ 

xexp(flog(2H + i^ 1 4 



We observe that 



nw„(o) 

[] (#>(&))! = exp (n^Tlog (*(&)!) u n (a, £) 

j=l I 



and hence 



< exp ( — n^^log (i(b)l) v n (a, £) + nw n (a, b) log (nza n (a, &)) — nw n {a,b) 



nw n (a, b) 
£a \b), j < nuo n {a) 

x ex P ( i2nJ„(a,6) + 2 log(27rn)) . 

Next, we obtain, 

n(a,b) 

^^„(aW & )-n^„(a)l {a=1 ,^ _ (fc _ ^ < ^ ^ &) bg pVy) 



fc=l 



x exp ( n(a, 6) log (l - - n ^" ( ( °ff n(6) ; 
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Putting everything together, recalling that H(tu) is the entropy of a discrete measure uj, we get 
F{M = v n \$(M) = (uj n ,m n )} 

< exp ( - nH(u) n ) + nH{v n ) - n Y ^(log^(6)!) i/ n (a,^) ro n (a,6) logro n (a,6) 

(a,b) I (a,b) 

-n^2 "&n(a, b) - § ^ tzj n (a, 6) log ((1 + 1{ 0=6 }) u n (a)u n (b)) + e[ n) ^ , 

(a,6) {a,b) 

for a sequence which does not depend on v n and satisfies lim n | 00 A = . To give the right 
hand side the form as stated in the theorem, we observe that 

H(u n ) - H(y n ) + Y £>g *(&)!) u n (a,i) - Y 

zu n (a, b) logro n (a, 6) + ro n (a, 6) 

(a.,6) £ (a,6) (a,6) 

+ I X] w n{a,b) log ((1 + l{ a=6 }) ^(0)^(6)) 
(0,6) 

= £ u n (a,i)[log, n (a,i) ~ log^(a) - £ (log (^f )^ - ^ - (log *(&)!))' 
= Y i/„(a, £) [ log „ n (a, £) - log (u, n (a) J] (~"(°- & >/""W>^ff>(-~ r^l/^M )' 

(a,£) 6 

= H(v n || Q„), 

which completes the proof of Lemma I4..SI ■ 

We can now complete the proof of the upper bound in Theorem 12.51 by combining Lemma 14.11 and 
Lemma POl Suppose that F C x N(X)) is a closed set. Then, 

p{Mer|$(M) = K,wn)}= Y p{m = i/„|$(m) = K,^ n )} 

y n ern/C(«)(a;„,ra7„) 

< tf/C^ rc7 n ) exp ( - n inf #(z^ n || Q n ) + sup e^^n)) . 

v i/„srnfc(»)(u>„,ra7„) i/„e/c(")(a)„,a7„) 7 

We have already seen that A sup^ e^(^ n ) and ~ log [J/C^(a;„, tu n ) converge to zero. It remains to 
check that 

Jim sup \H(u n || Q n ) - H(y n \\ Q)\ = 0. (4.6) 



To do this, we observe that 



|| Qn) ~ H(u n || Q) = Yl V n{a,£)\og-^^ 

= -H{u> n \\u)-H(w n \\m)- Y ro M)w + E ^M)log^ + |K||. (4.7) 

a,b£X a,b£X 

Note that this expression does not depend on u n . As the first, second and fourth term of Q4.7|) converge 
to 0, and the third and fifth term converge to ||ot||, the expression Q4.7j) vanishes in the limit, and this 
completes the proof of the upper bound in Theorem 12.51 
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4.3 An upper bound on the support of empirical neighbourhood measures 

The support, denoted S(u) C X x J\f(X), of an empirical neighbourhood measure v of a graph with n 
vertices is naturally bounded by n. For the proof of the lower bound in Theorem 12 . 51 we need a better 
upper bound. We still use m to denote the cardinality of X, and let 



C:=2 ...rV + 2)^ andD;=2 ,„ (■» + !)■ 



r(m) r(m) ' 

where T(-) is the Gamma function. 

Lemma 4.4. For every (ui n ,m n ) <G M n (X) x M n (X x X) and v n G M n (X x A/"(Af)) ${v n ) 
{oj n , zu n ), we have 



%S(v n )<c[n\\m n \\\ m+1 +D. (4.8) 

The following lemma provides a step in the proof of Lemma 14.41 
Lemma 4.5. Suppose j G N U {0} and n 6 N. Then, 

T^j 71 - 1 < • • • , *n) € (NU {0}) n : i! + . . . + Z„ = i} < ^ (j + n)"" 1 . (4.9) 

Proof. The proof is by induction on n. Equation ()4.9|) holds trivially for all j £ NU {0} and 

n = 1, 2, so we assume it holds for all j and n > 2. By the induction hypothesis, for any j, 



r n ~ ° n ~ 1 - £ H {lu ■ ■ ■ ' Zn - l} G (N u {0})n 1 : h + • ■ ■ + ln ~ l =j ~ 1 } 

' 3 Z=0 

= ${(h,..., l n ) € (N U {0}) n : + . . . + Z n = i} < -L ^(j - I + n) n -\ 

^ n > 1=0 

For the first and last term, we obtain the lower and upper bounds 



£0' - /i" ' > f .</' ; '/.V = -/ 7— 77./' 



r(n) 

n r(n + 1)' 



and 

- l + nT~ l < / y^dy < / y^dy = - (j + n + 1)" = ^ (j + n + If, 

Jn JO n 1 \ n + l ) 

which yields inequality (|4.9|) for n + 1 instead of n, and completes the induction. 

Proof of Lemma 14.41 Suppose (u n ,zu n ) £ M n (X) x M n (X x Af). Let 
Mi) :=K(«>*)e*xJV(*) : E^ 6 ) = J'} 



fee A 1 

m x 



»{(«!, U e (NU{0}) m : Zi + ...+z m =j}. 
For any positive integer A; we write 

9 k = min[e EN :J2 a m(j) > k}. 

j=0 
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We observe from Lemma 14.51 that. 

3=0 j=0 J0 

1_ 

Thus, we have > (fcr(m)) m — m =: a^. This yields 

Ei«4i) > rfe £i m > rfe P'V*/ > tw«j(«* " !) m+1 . ( 4 - 10 ) 

i=o i=o - 70 

where is the smallest integer greater or equal to y. 

Observe that the size of the support of the measure v n G JC^(uj n , w n ) satisfies 

Ok 

$S(v n ) <max|fc: y^Ja m {j) <n||ro n j||, 

3=0 

and hence, using (|4.1Uj) and the inequality (a + b) m < 2 m (a m + b m ) for a, b > 0, 
$S(v n ) < max (a k - l) m+1 < n||ro n ||| 

< r(m) _1 f (n||ro n ||)^+ir(m + 2)^+i +m+ 1J < C (n||zu n ||)™+l + L> , 
where the constants C, D were define before the formulation of the lemma. ■ 



4.4 Approximation by empirical neighbourhood measures 

Throughout this section we assume that tu n G A4 n (X) with u> n — > uj, w n G A4 n (X x ^) with w n — > ro, 
and that ^ G 3Jl[u;, ro]. Our aim is to show that ^ can be approximated in the weak topology by the 
empirical neighbourhood measure of a graph with n vertices, empirical colour measure u n , empirical 
pair measure u n , and the additional feature that the degree of any vertex is bounded by n 1 ' 3 . The 
approximation will be done in four steps. 

We denote by d the metric of total variation, i.e. 

d(v,u) = ± £ \u(a,£) -u(a,£)\, for u,u G M(X x N{X)). 

(a,e)exxj\f(x) 

This metric generates the weak topology. 

Lemma 4.6 (Approximation Step 1). For every e > 0, there exist v G A4(X x J\f(X)) and w G 
Ai(X x X ) smc/i t/iat \w(a, b) — w(a, b)\ < e for all a,b G X , d{v, v) < e and (w, v) is consistent. 

Proof. By our assumption (w, v) is sub-consistent. For any b G X define G M{X) by e <yb \a) = 
if a 7^ b, and e^ b \b) = 1. For large n define measures i> n G M(X x TV(A')) by 

^(a,Z) = KM) (l - HHMMI) + £ = ro(a ' b) - <K » Q4 - ))(a ' fe) • 

hex 

Note that z> n — > f and that, for all a, b G Af, 

£ ^(a,^(b) = (l- IH| - |K y W))l1 ) £ K«,W)+^(a,6)-(K-^)^(-))(«,&) 

t&]\f(X) ' feAf(#) 

= ro (a, 6) - IHHIM-A<(0)ll H,^(0>(a, 6) *3 ^(a, 6) . 
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Hence, defining w n by zn n (a,b) = Ylv n (a,£)£(b), we have a sequence of consistent pairs (vJ n ^ 
converging to (w,v), as required. □ 

Lemma 4.7 (Approximation Step 2). For every e > 0, there exists n(e) such that, for all n > n(e) 
there exists v n G M n (X x Af(X)) with Q{h> n ) = (i^n, ^n) such that d{y n , v) < e . 

The key to the construction of the measure v n is the following 'law of large numbers'. 

Lemma 4.8. For every 5 > 0, there exists v G M(X x M{X)) with d{y,v} < 5 such that, for 
i.i.d. J\f{X)-valued random variables £°j, j = 1, . . . , nuj n (a) with law u{ ■ \ a) := £({a} x -)/i>i(a), 
almost surely, 

nu> n (a) 

limsup(- £j(b) -vj n {a,b)) <5, foralla,beX. (4.11) 

Proof. By Lemma 14.61 we can choose a consistent pair (w, v) such that d{v,u) < 5 and, for all 
a, b G X, 

The random variables ij(b), j = 1, . . . ,nuj n (a) are i.i.d. with expectation 
Hence, by the strong law of large numbers, almost surely, 

nu) n {a) 

limsupf- T q(b)-w n (a,bj) <^uj(a,b)-w(a,b)<5, 
where we also used that uj n {a) — > uj{a) = vx(a) and w n (a, b) —* w(a, b). ■ 



Proof of Lemma 14. 7L We use a random construction. Given v G 5D?[u;,z<7] and e > 0, choose v 
as in Lemma 14.81 with 5 = e/(3m), where m is the cardinality of X. For every a G X, we draw tuples 
£j, j = 1, . . . , nu n (a) independently according to z>( • | a) and define e n (a, 6) by 

naj n (a) 

e n (a, b) := - f*(6) - ro„(a, 6), for all a,b <E X. 

n ^-^ J 

We modify the tuples (£? : j = 1, . . . , nu> n (a)) as follows: 

• If e n (a, b) < 0, we add an amount to the last element £^ ^(b) such that the modified tuple 
satisfies e n (a,b) = 0; 

• if e n {a,b) > 0, by Lemma f4.8l the 'overshoot' ne n (a,b) cannot exceed nd. We successively 
deduct one from the nonzero elements in £j(b), j = 1, . . . ,nu n (a) until the modified tuples 
satisfy e n (a, b) = 0; 

• if e n (a, b) = we do not modify £j(b). 

We denote by (£? : j = 1, . . . , nto n (a)) the tuples after all modifications. 
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For each a G X define probability measures A n ( • | a) and A n ( ■ | a) by 

1 

nuo n (a) 



nw„(a) 



respectively, 

nu„( a ) 

We define probability measures i> n G M n (X x J\f(X)) and i/ n € M n (X x A/"(^0) by v n (a,£) = 
uo n (a)A n (£ | a), respectively u n (a,£) = u> n (a)A n (£ \ a), for (a,£) G A? x N(X). Recall from our modifi- 
cation procedure that, in the worst case, we have changed nmS of the tuples. Thus, 

d(i>n, v n ) <m5 <\e. 

As a result of our modifications we have $(vn) = (w n , vj n ). We observe that, for all (a, £) G X x AA(,Y), 
the random variables 

= £},..., i{c n(a) = i} 

are independent Bernoulli random variables with success probability v{£ \ a) and hence, almost surely, 

lim A n (£\a) = v(£\a) . 

Therefore, for all (a,£) G X xJ\f(X), we obtain lim^oo v n (a,£) = u(a,£), almost surely. Thus, almost 
surely, for all large n, we have d{v n , v) < d{y n% u n ) + d(i> n , v) + d(z>, u) < e, as claimed. ■ 

Lemma 4.9 (Approximation Step 3). Let u n G M. n {X x M{X)) with <&(v n ) = (uj n ,w n ). For ev- 
ery e > there exists n(e) such that, for all n > n{e), we can find v n G M. n {X x M(X)) with 
&(v n ) = {w n ,m n ) and d(v n ,v n ) < e, such that 

E £{b) < n 1//3 for v n -almost every (a,£). (4-12) 
fee A" 

Proof. As v n G M. n {X x J\f(X)), there is a representation 

1 n 

z/ n = - ^«5( afc ,4), for a fe G AT, 4 G jV(Af). 
n fe=i 

Fix 5 > and a £ X. Look at the sets 

. y+ = {1 < k < n: a k = a, £ b 4(&) > n 1 ^} wit h %V+ < (n £ 6 tu n (a, 6)) 2 / 3 , 
. y- = {1 < k < n: a fc = a, £ b 4(&) < ™ 1/4 } with $V~ > n - (^^n(o,6)) 3 / 4 . 

For each k G V\ we replace 4 by a smaller vector 4 such that ^2 b £k(b) = n 1 / 3 . As 

E E^ fe ( 6 ) - n E ron ( a ' 6 ) 
fceVi fe fee A" 

we replace (for large n) no more than Sn of the vectors 4 5 k G V2, by larger vectors 4 such that 

n n 

< n 1 / 3 and E E l{a fc = a}4(&) = E E ^ = > 
b k=ibex k=ibex 
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where we use the convention if. = if* if this vector was not changed in the procedure. Performing such 
an operation for every a G X we may define 

1 n 

k=l 

and observe that (|4.12|) holds and Q(v ri ) = {uj n ,m n ). Moreover, 

b)) 2 / 3 + Sn), 

b 

which is less than e > for a suitable choice of S > 0, and all sufficiently large n. ■ 



Lemma 4.10 (Approximation Step 4). Let v n G M n (X x J\f(X)) with <&(v n ) = (uj n ,va n ). For 
every e > f/iere exists n(e) such that, for all n > n{e), we can find v n G JC^ (u> n ,w n ) with d(v n , u n ) < 
e such that Q4.12JI holds. 

Proof. By Lemma 14.91 we may assume that v n can be represented as 
1 n 

v n = - X^KA)> for a k ^X,£ k e M(X) with < ™ 1/3 • 

fc=l be* 

For the proof it suffices to construct, for every e > and all large n, a (coloured) random graph X 
with n vertices such that 

limsupP{d(i/„,Mpf)) > e] = 0, (4.13) 

n^oo 

We now describe the random procedure that generates X. First, equip each vertex with an element 
of X x M{X) by drawing without replacement from the collection {(a%,£i), . . . , (a n ,i n )}. We denote 
by V(a) the collection of vertices which have colour a G X, an observe that $V(a) = nu) n {a). 

Now fix a, b G X. We construct two collections of vertices: If a vertex is equipped with (ak,ik) and 
afc = a, then it is represented ik{b) times in the first collection, W(a). If at = b, then it is represented 
£k(b) times in the second collection W(b). Hence there are exactly nw n (a,b)/2 vertices of colour a in 
collection W(a), and exactly nzu n (a,b)/2 vertices of colour b in collection W(b). 

We now match vertices from the two collections randomly: At each step k = 1, . . . ,nu7 n (a,b)/2, we 
randomly pick two vertices V* G W(a) and V* G W{b). We connect and V* by an edge unless 
Vi = VJ? or the two vertices are already connected. If one of these two things happen, then we simply 
choose an edge randomly from the set of all possible edges connecting colours a and b, which are not 
yet present in the graph and whose introduction does not violate 1)4.12(1 . 

This completes the construction of a graph with L X (X) = u> n , L?(X) = w n and 

d{v n ,M{X))<l Y, B >> 6 )> ( 4 - 14 ) 

a,b£X 

where B n (a, b) is the total number of steps k G {1, . . . , nzn n (a, b)/2} at which there is disparity between 
the vertices , drawn and the vertices which formed the fc th edge connecting a and b in the random 
graph construction. 

Given a,b G X, using Yl^k(b) < n 1 / 3 , the probability that = VJ? or the two vertices are already 
connected is less or equal 

n <~ ^ — 2 " 1/3 H _I_ f 1 2nV 3 n \ 4(fc-l)n 2 / 3 

— n ro „(a,fe) 1 {a=fc} + I 1 - n row (a,h) ^{a^jj (n ro „(a,fc)) 2 ' 
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B n (a, b) is a sum of independent Bernoulli random variables X\, . . . , X nvJn u b y 2 with 'success' prob- 
abilities less or equal p\i](a, b), . . . ,P[ n zn n (a,b)/2]( a i b). So its expectation satisfies 

nuj n (a,b) /2 



k=l 



Hence, we have that Vai(B n (a, b)) < KB n (a,b) = o(n). By Bernstein's inequality, see for exam- 
ple |BBL041 Theorem 3], we obtain, for any e > 0, 

F{B n (a, b) > EB>, b) + 2ne} < exp { - 2(yat ^ ))+ne/3) } • 

Using the bounds on ~EB n (a, b) and V&r(B n (a, b)), we obtain, for sufficiently large n, 

F{B n (a, b) > o(n) + 2ne} < exp { - 2o(n ff n ./ 3 } ■ 

Now suppose a small e > and large A > are given. Let 5 = e/(2m 2 ). Suppose that B n (a, b) < nS, 
for all a,b£ X. Then, by (l4~T4l . d(Mpf), i/ n ) < 25m 2 = e. Hence, 



m 2 sup P{B n (a, b) > o(n) + (n<5)/2} 
<m 2 exp{ - 8o(n) ^ )/3 }- 



This completes the proof of the lemma. ■ 

4.5 Proof of the lower bound in Theorem 12.51 

There is a partial analogue to Lemma 14.31 for the lower bounds. 

Lemma 4.11. For any sequence (u n ) with v n € K,^ (uj n , vu n ), which satisfies 1)4.12)1 . and e > 0, we 

have 

P{d{M, u n )<e\ $(Af) = K, w n )} > exp ( - nH{y n \\ Q n ) - e { 2 n \v n )) , 
where Q n is as Lemma \4-<A an d 

lim±4% n ) = 0. 

n\oo 

Proof of Lemma 14. Ill We use the notation and some results from the proof of the upper bound, 
Lemma 14.31 Recall that 

P{M = I/„ $(M) = (0Jn, KJn)} = — ,r. , , 777 , 

|{lGAi(w n ,C7„))} 

and that the denominator was evaluated in ()4.3)) as 

. n(a,b) 

U I TT TT ( n2u) ri(a)u n (b)-nuj n (a)l {a=b} . ,x 

v V ; ' 7 {a,fe} fc=l 

The numerator can be estimated from below by assigning edge-numbers to the vertices in a manner 
cautious to avoid loops and double edges. In each step the number of assignments, which lead to 
multiple edges or loops, is bounded by the square of the maximal vertex degree and hence by n 2 / 3 . 
Hence the numerator is bounded from below by 

n \ tt (nw n (a,b) - n 2 / 3 )! 2 _n ron(A) 

nu n (a,£), aeXJe N{X)J 11 H™^ a \$ >(&))) 
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We again use Stirling's formula as in the proof of the upper bound. For the denominator we get the 
same main terms as in Lemma 14.^1 with slightly different error terms, which however do not depend 
on v n . More interestingly, we have 

]T( ( ^^fl \riv\ I > exp (nVw n (a)logw„(a) - nV i/ n (a,£)logi/ n (a,f) 
i A „ \nv n {a,l), I G N{X)J \ 



x exp ( - log(2vrn) - l - 



12nu n (a,£) J ' 

nv n (a,£) > 1 

where the exponent in the error term is of order o(n), by the bound on the size of the support of v n 
given in Lemma 14.41 Further, 

ra " n( ^ (j) 1 > exp [ - n^log (£{b)\) u n (a,t) + nw n (a, b) log (nuj n (a,b)) - nw n (a,b) 

Y[j=l (^a (P))\ t 

x exp ( - n 2/3 log {nw n {a, b)) + nw n {a, b) log (l - n ^^ b) 
and the result follows by combining this with facts discussed in the context of the upper bound. ■ 

To complete the proof of the lower bound in Theorem 12. 51 take an open set V C A4(X xAf(X)). Then, 
for any v G r n 97T[u;, w\ we may find e > with the ball around u of radius 2e > contained in T. By 
our approximation, Lemmas 14.71 and I4.1U1 we may find v n G T D K,^ n \u) n ,w n ) with d(u n ,u) { such 
that (|4.12|) holds. Hence, for all large n > n(e), 

p{m g r | $(M) = K,ro n )} > ¥{d(u n ,M) < e | $(M) = K,ro n )} 

> exp ( - ni?(^ n || Q n ) + e^ n) (^ n )^ . 

We observe that 

lim H(v n || Q n ) - H{u || Q) = lim £T(i/ n || Q n ) - H(u n \\ Q) + lim H(y n \\ Q) - H{y \\ Q) = 0, 

n— >oo n— >oo n— >oo 

where the last term vanishes by continuity of the entropy, and the first term was shown to vanish in 
the proof of Lemma 14.111 As v G r n 971 [a;, w] was arbitrary, this completes the proof of Theorem 12.51 

5. Proof of Theorem 12.11 by mixing 
We denote by 6 n := M n (X) x M n (X x X) and G := M(X) x M*(X x X). With 

P (ltw n )K) := HM = v n | 3(M) = K,t*„)} , 

i>( n V n ,ts7 n ) :=F{(L\L 2 ) = (u n ,w n )} 
the joint distribution of L 1 , L 2 and M is the mixture of P, , with P^ n Hoj n . w n ) defined as 

dP n (cu n ,w n ,u n ) := dPjri >Wn) (u n )dPV)(u n ,™ n ) • (5-1) 

Biggins Bi04, Theorem 5(b)] gives criteria for the validity of large deviation principles for the mixtures 
and for the goodness of the rate function if individual large deviation principles are known. The 
following two lemmas ensure validity of these conditions. 

Lemma 5.1. (P n : n G N) is exponentially tight. 
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Proof. Given k G N, we observe from Lemma 13.21 that there exists N(k) € N such that, for all 
sufficiently large n, 

P{M({||*|| > 2kN{k)}) > k- 1 or ||L 2 || > 2N(k)} < F{\E\ > nN(k)} < e~ kn . 

Now, for any 9 > 0, we define the set Eg by 

Eg := {{m,v) € M*(X x X) x M{X x Af(X)) : u{\\£\\ > 2lN(l)} < > 9 and \\w\\ < 2N(9)}. 

As {\\£\\ < 2lN(l)} C M(X) is finite, hence compact, the set Eg is relatively compact in the weak 
topology, by Prohorov's criterion. Moreover, we have that 

oo 

P^((E g ) c ) < P{||L 2 || > 2N(6)} + J2^{M({\\£\\ > 2lN(l)}) > r 1 } < C(9)e~ ne . 
Therefore, limsup - logP n ((clH6)) c ) < —9, which completes the proof, as 9 > was arbitrary. ■ 

n—roo 

Define the function 

J: 6 x M(X x N(X)) - [0, oo], J((u,w), v) = J (WjB7 )(i/). 
Lemma 5.2. J is lower semicontinuous. 



Proof. Suppose 9 n := ((w„,w n ), v n ) converges to 9 := ((u),m), v) in x Ai(X xAf(X)). There is 
nothing to show if liminfe n ^g J(9 n ) = oo. Otherwise, if {w n , v n ) is sub-consistent for infinitely many 
n, then 

w{a,b) = \imm n (a,b) >limmf(v n (-,£),£(-))(a,b) > (v (■,£),£(■)) (a, b) , 

n|oo nfoo 

hence (w, v) is sub-consistent. Similarly, if the first marginal of v n is uj n , we see that the first marginal 
of v is oj. We may therefore argue as in (|4.6j) to obtain 

liminf J(9 n ) = liminf H{v n \\Q n ) > lim H(u n \\ Q n ) - H(u n \\ Q) + liminf H{y n \\Q) = H(v\\Q), 
where the last step is because of continuity of the entropy. This proves the lemma. ■ 



By |Bi041 Theorem 5(b)] the two previous lemmas and the large deviation principles we have estab- 
lished in Theorem 12 . 31 and 12.51 ensure that under (P n ) the random variables (iv n , VJ n , v n ) satisfy a large 
deviation principle on M.(X) x M.*(X x X) x M(X x M(X)) with good rate function 



J(cj, w, u) 



H{u || n) + I $)c{™ II w) + H(u ||Q) , if v G 0%, w] 
oo , otherwise. 



By projection onto the last two components we obtain the large deviation principle as stated in 
Theorem 12.11 from the contraction principle, see e.g. |DZ981 Theorem 4.2.1]. 



6. Proof of Corollaries 12.21 and 12.41 



We derive the corollaries from Theorem 12.11 bv applying the contraction principle, see e.g. fpZ98, 
Theorem 4.2.1]. It just remains to simplify the rate functions. 
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6.1 Proof of Corollary 12.21 In the case of an uncoloured Erdos-Renyi graph, the function C 
degenerates to a constant c, I? = \E\/n £ [0, oo) and M = D £ A4(N U {0}). Theorem 12.11 and the 
contraction principle imply a large deviation principle for D with good rate function 

6(d) = inf {J(x, d) : x > 0} = inf {H(d || q x ) + ^xlogx — ^xlogc + \ c — ^x: (d) < x }, 

where q x is the Poisson distribution with parameter x. We denote by S x (d) the expression inside the 
infimum and consider the cases (i) (d) > c and (ii) (d) < c separately. 

Case (i): For any e > 0, we have 

^(d) - = § + ^log^ + flogM > | + (^b£ (-) + |i og M > o, 

so that the minimum is attained at x = (d). 

Case (ii): Under our condition the equation x = ce~ 2 ^~^' x ' has a unique solution, which satisfies 
x > (d). Elementary calculus shows that the global minimum of y t— > 5 y (d) on (0, oo) is attained at 
the value y = x, where x is the solution of our equation. 

6.2 Proof of Corollary 12.41 We begin the proof by defining the continuous linear map W : M(X) x 
Ai(X x X) — > [0, oo) by W(uj,w) = *\\vj\\. We infer from Theorem 12.31 and the contraction principle 
that W(L ,L 2 ) = \E\/n satisfies a large deviation principle in [0, oo) with the good rate function 

((x) = inf \I(u>, w) \ W(co,zu) = x\ . 

To obtain the form of the rate in the corollary, the infimum is reformulated as unconstrained optimi- 
sation problem (by normalising w) 

inf {h(uj II u) + xH(w II Ceo ® u) + xlog2x + 1 \\Cuj ® u\\ - x\. (6.1) 

By Jensen's inequality H(w \\ Cuj <8> cj) > — log ||Cu; (8> w||, with equality if w = jj^§^jj, and hence, by 
symmetry of C we have 

min \ H(uj \\ u) + xH(w \\ Cu> ® w) + x log 2x + i NCtJ ® o;|| — x > 
= H(uj || /i) — x log || Coj ® a; 1 1 + x log 2x + i ||Ca; ® a;|| — x. 
The form given in Corollary 12.41 follows by defining y = J2 a bex C( a > b)u(a)uj(b). 
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